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In statistical mechanics, we consider quantum mechanical 
systems in which particles like atoms and electrons have 
different energy levels. 

We want to know the probability that the particles are at each 
level. We can use this information to calculate the population 
at each level, the total energy of the system, the heat capacity, 
etc. 

Later on, we shall learn how to calculate this probability using 
the Boltzmann distribution. Before that, we need to answer 
one question: 

Even if we know the probabilities at each level, particles are 
often free to hop between levels. Would this not cause big 
changes in energy of the system? 

In order to answer this question, we investigate the example 
where we only have 2 energy levels, and with equal probabilities. 
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To use a familiar example, think of each particle as a coin and 
each level as one side of the coin. 

Suppose the coin is tossed 2 times. The possible outcomes are: 

TT 

HT 

TH 

HH 

Let x be the number of heads. Let n be the total number of 
outcomes. 

Let P(x) be probability of getting x heads. This is given by 

P(x) = x/n. 

p(0) = 1/4 
P(l) = 2/4 
P(2) = 1/4 
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Mean and standard deviation 



Suppose we toss the coin n times and record the number of H. 
Suppose we repeat this many times. 

We would get a sample of data for x, e.g. {2, 0, 1, 3, 2, 2, 3, 
0, ... } 

In statistics, we are often interested in two numbers - the mean 
and the standard deviation. 

The mean is the simple average. 

The standard deviation tells us how much the numbers in the 
data deviate from the mean. 
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If we know the probaility distribution P(x), we can also 
calculate the mean and standard deviation without actually 
tossing the coin. 

The mean is given by 

/i = xP(x). 



The standard deviation is the root mean square of x — \i. So it 
is essentially a kind of average of how much x deviates from \±. 

It is given by 
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When the standard deviation is small 



The standard deviation is relevant to the idea of macrostates 
that we want to develop. 

We are particularly interested in what it means if the standard 
deviation becomes very small. 

Recall that the standard deviation is a measure of the average 
deviation of x from the mean. 

So if the standard deviation is small, it means that most of the 
data is very close to the mean value. 



Large standard deviation 

P(x) 



Small standard deviation 



P(x) 



Etu=- 



If we look at a histogram of the data, it would look more 
sharply peaked. 
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The binomial distribution 



Let p be probability of getting a Head, and q the probaility of 
getting a Tail. 

If we have a fair coin, then p = q = 1/2. 
Suppose we toss a coin N times. 

The probaility of getting x heads is given by the binomial 
distribution: 

P( x ) = N C n p n q N - n . 
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Mean and standard deviation 



When we toss a coin N times, suppose that we get x heads. 

When we toss TV times again, we may get a different x. 

The mean value of x is given by a sum over x, weighted by the 
probabilities: 

N 

\± = ^2 xP{x) — Np. 
The standard deviation is given by: 



Both formulae can be derived by substituting the binomial 
distribution P(x) = N C n p n q N ~ n . 



N 
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Distribution of the number of Heads 



Lets get some idea of how a changes a TV is increased. For a 
fair coin p = q = 1/2. 



40000 100 
4000000 1000 

It looks like the number of heads changes more from one toss 
to the next, as the number of coins is increased. 

But if we plot the distribution on a graph: 
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we would find that it gets more sharply peaked! 
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A reference quantity 



The reason is because, on the graph, we are looking at the 
standard deviation relative to the total number of coins N. 

So, although 10 is large than 1, 10/400 would be smaller than 
1/4. 

In the case of the coins, we should therefore be comparing the 
standard deviation with the total number of coins N. 

x is the actual number of heads. 

The average of deviation of this number from the mean, as a 
fraction of TV, is a/N = ^pq/N. 

When TV is very large, this fraction approaches zero. 
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The dotted curve is the smallest N, the solid curve the largest 
N. Notice that the peak gets narrower for large N. 

If TV is as big as the number of atoms in a real material, the 
peak becomes extremely narrow. 

Then x = N/2 becomes almost the only possible value. 
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The Analogy with Temperature 



We can apply this to atoms: 

1. H is the higher energy state, T is the lower one. 

2. One toss of a coin is one atom. 

3. The larger the number of H, the higher the temperature. 

In a real material at room temperature, atoms could hop 
between energy levels randomly through colliding with other 
atoms. 

Because the number of atoms is huge, deviation of particle 
numbers at each level from the values predicted from their 
probabilities are extremely small. 
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Warming up exercises 
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Mole Concept 



Exercise 1 

How many atoms are there in each of the following? 
(N A = 6.022 x 10 23 ) 

1) 1 mole of iron 

2) 0.2 mole of copper 

3) 3 moles of silver 

4) 1 mole of oxygen (O2), and how many molecules? 

5) 1 mole of water (H2O), and how many oxygen atoms? 

6) 1 mole of sodium chloride (NaCI), and how many sodium 
ions? 

7) 1 mole of helium-4 ( 4 He) 

8) 1 mole of helium-3 ( 3 He) 
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Mole Concept 



Solutions to Exercise 1 



6.022 x 10 23 

0.2 x 6.022 x 10 23 

3 x 6.022 x 10 23 

2 x 6.022 x 10 23 atoms, and 6.022 x 10 23 molecules 

3 x 6.022 x 10 23 atoms, and 6.022 x 10 23 oxygen atoms 
2 x 6.022 x 10 23 atoms, and 6.022 x 10 23 sodium ions 
6.022 x 10 23 

6.022 x 10 23 



15 



Mole Concept 



Exercise 2 

What is the mass of each of the following? (A r stands for 
relative atomic mass.) 

1) 1 mole of iron (A r =56) 

2) 0.2 mole of copper (A r =64) 

3) 3 moles of silver (A r =108) 

4) 1 mole of oxygen (O2) (A r =16) 

5) 1 mole of water (H 2 0) (A r of H=l, of 0=16) 

6) 1 mole of sodium chloride (NaCI) (A r of Na=23, of 
Cl=35.5) 

7) 1 mole of helium-4 ( 4 He) 

8) 1 mole of helium-3 ( 3 He) 
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Mole Concept 



Solutions to Exercise 2 

1) 56 g 

2) 0.2 x 64 g 

3) 3 x 108 g 

4) 2 x 16 g 

5) (2x1 + 16) g 

6) (23 + 35.5) g 

7) 4 g 

8) 3 g 
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Mole Concept 



Exercise 3 

How many atoms are there in each of the following? (A r 
stands for relative atomic mass.) 

1) 1 g of iron (A r =56) 

2) 1 g of copper (A r =64) 

3) 1 g of silver (A r =108) 

4) 1 g of oxygen (0 2 ) (A r =16) 

5) 1 g of water (H 2 0) (A r of H=l, of 0=16) 

6) 1 g of sodium chloride (NaCI) (A r of Na=23, of Cl=35.5) 

7) 1 g of helium-4 ( 4 He) 

8) 1 g of helium-3 ( 3 He) 
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Mole Concept 



Solutions to Exercise 3 



1) 1/56 x 6.022 x 10 23 

2) 1/64 x 6.022 x 10 23 

3) 1/108 x 6.022 x 10 23 

4) 1/(2 x 16) x 6.022 x 10 23 

5) 1/(1 + 1 + 16) x 6.022 x 10 23 

6) 1/(23 + 35.5) x 6.022 x 10 23 

7) 1/4 x 6.022 x 10 23 

8) 1/3 x 6.022 x 10 23 
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Some exercises 



Exercise 4 

Suppose a biased coin is tossed 2 times. The probability 
distribution for 0, 1 and 2 heads are given by 0.36, 0.48 and 
0.16, respectively. Find the mean and standard deviation of the 
number of heads. 

Answer 

Let x be the number of heads. The distribution given is 

x P(x) 

0 0.36 

1 0.48 

2 0.16 
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Some exercises 



The mean is \i = Y,xP{x) = 0 x 0.36 + 1 x 0.48 + 2 x 0.16 = 0.8. 

(Out of 2 tosses, we only get 0.8 heads on average. Obviously, 
this coin is biased towards the tail.) 

x x — \± ( x — /x) 2 

0 -0.8 0.64 

1 0.2 0.04 

2 1.2 1.44 

The standard deviation is 

= V0.64 x 0.36 + 0.04 x 0.48 + 1.44 x 0.16 = 0.48. 
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Microstates and Macrostates 
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Consider a particle with 2 energy levels: e\ and S2- Suppose we 
have N = 2 particles. And there are n\ particles at energy level 
ei, and ri2 at S2- 

The following lists all the possible arrangements: 
M a crosta tes M i crostates 



For example, [ei,£2] would mean that particle 1 is at level s\ 
and particle 2 is at level S2- This is one particular arrangement. 

A particular arrangement of particles among energy levels is 
called a microstate. 



(2, 0) 
(0, 2) 
(1, 1) 



[^2,^2] 

[£l,£ 2 L [£2,£l] 
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(ni,ri2) specifying the number of atoms at each level. 

The set of numbers specifying the number of atoms at each 
level is called the macrostate. 

E.g. Consider a particle with 3 energy levels: si, S2 and S3. 
Suppose there are N = 2 particles. And there are n\ = 2 at 
energy level si, 712 = 0 at S2, and 713 = 0 at S3. 

The macrostate in this case is (711,712,713) = (2, 0, 0). 

Lets write down a useful relation: 

N = n\ + 712 H~ n 3 

The total number of particles is equal to the sum of the 
particles at each level. 
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Particles with more than two energy levels 



In general, a system can have many energy levels. We want to 
know the most likely macrostate (ni, 712 > 77,3, ...) of a system, 
because this would allows us to find the total energy, heat 
capacity, etc. 

The most likely macrostate is the one with the largest number 
of microstates - or arrangements. 

For two energy levels, the number of microstates for the 
macrostate {n\,n^) is 

n\ !n2' 

It the number of ways of putting TV different objects into 2 
boxes. 
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Particles with more than two energy levels 



For more energy levels, the formula generalises to: 



or 

TV" 

Q = 



The total number of particles is N. The macrostate is specified 
by the number of particles at each energy level: 
(ni, 7i2, n h •••)■ So the total number must the the sum of the 
particles at each level: 



Tin 



We consider an isolated system that is very well insulated from 
the surrounding. Then the total energy U of the particles 
would remain constant. 
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Particles with more than two energy levels 



There are n\ particles at energy level e\, so the total energy of 
these particles is n\s\. Adding up the energy of the particles at 
all energy levels, we have the energy of the whole system: 

U = n i 6 i 
i 

Q is a function of the variables (ni,n2,...). 

There are 2 conditions, or constraints, on these variables: 

(i) the total number of particles is a constant, and 

(ii) the total energy is also a constant. 

In order to maximise Q, we can use the method of Lagrange 
multipliers. 
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To find the most probable macrostate 
The procedure is as follows: 

1. We maximise InQ instead of Q. The result would be the 
same because when InQ is a maximum, so is Q. 

2. The reason for using InQ is that it can be simplified for very 
large TV using the Stirling's theorem. 

3. Write down the Lagrange function In Q + AiTV + X2U. 

4. a and /3 are new parameters called Lagrange multipliers. 

5. Differentiate this with respect to every r^. 

6. Set the derivative to zero and solve for n it Ai and A2. 

7. This gives the macrostate (711,712, ...)■ 
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Stirling's Theorem 



The Stirling's Theorem is used for large factorials. 

For real objects, TV may be the number of atoms. This could 
be as big as 10 30 , and it takes a long time to calculate. 

Fortunately, there is a simple, approximate formula that is very 
accurate for large numbers: 

In TV! « 7Vln N - N 
This formula is the Stirling's Theorem. 
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The algebra 



We need to maximise 

\nn + X 1 N + X 2 U 
In order to do so, we need to differentiate this. Start with 

N = n\ + n2 + ... + ni + ... 

Differentiating with respect to ni gives 

dN _ 



'i 

Next, we differentiate 

JJ = mei + n 2 C2 + ••• + riiei + 

This gives 



dU 
drii 



Next we need to differentiate InQ 
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The algebra 



Start with the formula 

TV' 

Q = ^' — — 
Then take the logarithm, 

In Q = In TV! — In ni! — In — ... — In — ... 
and apply Stirling's theorem: 

In Q = TV In TV— TV— (ni In n\ — n\) — (n<2 In n^—n^) — ... — {ni In n^— nj — ... 
Differentiating, we get 

<9lnQ 

= — in rii 

dni 

Note that in this differentiation, TV in Q is to be treated as a 
constant. This is unlike in AiTV where it is a function of n^. 
The reason is because Q is up to us to define, and it is 
sufficient to vary n^. 
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The algebra 



These are what we have obtained 

ON 



drii 
dU 

drii 
<9lnQ 

drii 



= 1 



= €i 



= —\nn 7 



To maximise the Lagrange function In Q + XiN + \2U, we need 

to differentiate this and set the derivative to zero: 

<9lnQ dN dU 
h Ax h A 2 = 0 

oni oni oni 

Substituting the above results, we get: 

- Inr^ + Ai + X 2 £i = 0 
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The most probable macrostate 



We have obtained 

- Inn* + Ax + X 2 £i = 0 
Soving for ni gives the solution: 

rii = Aexp(A 2 eJ 

where A = exp(Ai). 

Next, we need to determine Ai, or A, and A 2 . 
A can be obtained using 

the condition that the total number of particles is N. This is 
called the normalisation condition. Substituting, we get 

N = A^exp(A 2 e < ). 

Rearranging this gives 

A= N 

Eexp(A 2 €j) 
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Boltzmann's postulate 



Next, we need to find A2. In principle, we can use the next 
constraint 

Substituting the solution 

rii = Aexp(A 2 q) 

we get 

U = J2 Ae i exp(A 2 q) 

Then solve for X 2 . 

Unfortunately, this equation is very difficult to solve. Instead, 
we need to take another approach. That is to make use of our 
knowledge of thermodynamics. 
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Boltzmann Postulate 



35 



Boltzmann's postulate 



The purpose of doing all these calculations is to find a way to 
calculate thermodynamic quantities like entropy and heat from 
first principles. 



Instead of solving for A2 directly, we can try and find it in terms 
of known quantities in thermodynamics. We shall start by 
stating the result: 



where kg is the Boltzmann constant and T is the temperature. 
There are different ways to determine this. We shall look at 
the following approach. 
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Boltzmann's postulate 



To relate A2 to the entropy formula 

dQ = dU = TdS, 
we look at the Lagrange function In Q + \\N + X 2 U again. 

Previously, we have taken the derivative to find the set of 
numbers (711,712,...) that would maximise the function. We 
have found the solution for n iy except for the unknown A2. 

For small variations about this solution (711,712, ...)• the change 
in the Lagrange function would be zero, since it is at a 
maximum. So 

d(ln Q) + \\dN + X 2 dU = 0 

Suppose we impose the condition that the total TV is fixed, but 
we give some heat to the system so that U increases a little. 
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Boltzmann's postulate 



Since TV is constant, dN = 0. So 

d(ln Q) + XidN + X 2 dU = 0 

becomes 

= -— d(lnfi) 

A 2 

Compare with 

(it/ = Td5. 
Boltzmann postulated that 

S = k B \nn 

where k B is a constant. Substituting this into the above, we 
find that 



k B T 

This completes the solution for the macrostate (ni,n2,...) 



rii = Aexp(\ 2 £i) = Aexp ( - 
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Boltzmann's postulate 



We shall see later, in the lectures on ideal gas, that k is the 
Boltzmann constant 

R 

k B = = 1.3807 x 10~ 23 J K -1 , 

N A 

where R is the ideal gas constant and N A is Avogadro's 
constant. 

The solution for the macrostate tells us how the number of 
particles are distributed in different energy levels: 

rij — A exp I — — 

V k B T 

It is called the Boltzmann distribution. 
Historically, the formula 

S = kg In Q. 
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Boltzmann's postulate 



was first postulated by Boltzmann. It is so important that it is 
engraved on his tombstone ... 
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Worked Examples 
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Example 1 

Minimise the function f(x, y) = x 2 + y 2 , subject to the 
constraint x + y — 2 = 0. 

(i) Do it mentally, or by inspection of the graphs. 

(ii) Do it using Lagrange multiplier. 
Do the results agree? 
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Mentally 



Answer 




1. The contours of f(x, y) are just circles, e.g. x 2 + y 2 = l, 

x 2 + y 2 = 2, ... The circles get larger as the value increases. 

2. The constraint x + y — 2 = 0 is a straight line with 
intercepts x = 2 or y = 2. 
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Answer: Mentally ... 




1. By symmetry, f(x,y) is highest at the midpoint between the 
intercepts, where x = y = 1. 

2. Therefore, f(x,y) = l 2 + l 2 = 2. 
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Using Lagrange multiplier 



1. Let g{x, y) = x + y — 2. The constraint is then g(x, y) = 0. 

2. The Lagrange function is 

fix, y) + \g(x, y)=x 2 + y 2 + \(x + y- 2) (1) 

3. The gradient gives: 

1. Differentiate wrt x: 2x + X = 0. 

2. Differentiate wrt y: 2y + X = 0. 

4. Solving with the constraint equation gives X = —2 and 
x — y — 1, as before. 
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Example 2 



Use your calculator to work out In 10!. Compare your answer 
with the simple version of Stirling's theorem (TV In TV — TV). How 
big must TV be for the simple version of Stirling's theorem to be 
correct to within (ii) 5% (ii) 1% ? 

Answer 

ln(10!) = 15.11 whereas 10ln(10)-10 = 13.03. There is less 
than 5% difference for N = 24 and less than 1% for B = 91. 
Stirling's approximation is quite accurate even for relatively 
small N. 
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Boltzmann Distribution 



Example 3 

A system of particles is at a temperature of 2 K. Find the 
Boltzmann factor for each of these energy levels: 

(k B = 1.38 x icr 23 ) 



1) 


1 x 10 


-23 


J 


2) 


2 x 10 


-23 


J 


3) 


3 x 10 


-23 


J 


4) 


4 x 10 


-23 


J 


5) 


5 x 10 


-23 


J 


6) 


6 x 10 


-23 


J 


7) 


7 x 10 


-23 


J 


8) 


8 x 10 


-23 


J 



47 



Boltzmann Distribution 



Solutions to Example 3 



0.6961 

0.4845 

0.3372 

0.2347 

0.1634 

0.1137 

0.07916 

0.05510 
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Boltzmann Distribution 



Example 4 



There are lCr^ particles. The system has two energy levels 
only: 0 J and 10 -23 J. Find the number of particles at 0 J 
when the temperature is: 



1) 1 K 

2) 2 K 

3) 3 K 

4) 4 K 

5) 5 K 

6) 6 K 

7) 7 K 

8) 8 K 
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Boltzmann Distribution 



Solutions to Example 4 



1) 


6.736 x 


10 22 


2) 


5.896 x 


10 22 


3) 


5.601 x 


10 22 


4) 


5.452 x 


10 22 


5) 


5.362 x 


10 22 


6) 


5.302 x 


10 22 


7) 


5.259 x 


10 22 


8) 


5.226 x 


10 22 
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Example 5 



Consider a 10000 distinguishable particles at room temperature, 
298 K. Suppose that each particle has 2 energy levels, 0.01 eV 
and 0.02 eV. Find the number of the particles in each energy 
level. ( Boltzmann's constant is 1.3807 x 10 -23 J K _1 .) 
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Example 5 



Answer 

Since the number is quite large, we assume that the probability 
at each energy level is given by Boltzmann's distribution 

Aexv(-s/k B T). 

At 0.01 eV, Aexp(-s/k B T) = 0.6778A 
At 0.21 eV, Aexp(-s/k B T) = 0.4594A 
The total is 0.6778A + 0.4594A = 10000. So A = 8794. 

Therefore the numbers are: 

at O.OleV, 0.6778 x 8794 = 5961; 
at 0.02eV, 0.4594 x 8794 = 4040. 
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Example 6 



Calculate the quantity k B T at room temperature, 298 K. Give 
the answer in eV. ( Boltzmann's constant is 1.3807 x 10 -23 J 
K _1 . Electron charge is 1.6 x 10 -19 C.) 

Answer 

In Joules, k B T = 1.3807 x 10~ 23 x 298 = 4.114 x 10~ 21 J. 

In eV, k B T/e = 4.114 x 10- 21 /:L6 x 10~ 19 = 0.026 eV. 

Note: Since 1/40 = 0.025 is quite close to this answer, k B T at 
room temperature is often quoted as 1/40 eV. 
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Example 7 



Consider a large number of distinguishable particles, at 
temperature T.. Each particle has 4 energy levels 0, k B T, 2k B T 
and 3k B T. Calculate the fraction of particles in each energy 
level. 

Answer 

Since the number is quite large, we assume that the probability 
at each energy level is given by Boltzmann's distribution 

Aexp(-s/k B T). 
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First, calculate the following: 

at energy 0, exp(—e/k B T) = exp(O) = 1. 
at energy k B T, exp(-s/k B T) = exp(-l) = 0.3679. 
at energy 2k B T, exp(-s/k B T) = exp(-2) = 0.1353. 
at energy 3k B T, exp(-s/k B T) = exp(-3) = 0.0498. 

The sum is 1.553. The fractions are, therefore: 

at energy 0, 1/1.553 = 0.6439. 
at energy k B T, 0.3679/1.553 = 0.2369. 
at energy 2k B T, 0.1353/1.553 = 0.0871. 
at energy 3k B T, 0.0498/1.553 = 0.0321. 

Notice that fraction drops to nearly 1/10 after just 2k B T. 

In statistical mechanics, we often want to know if an energy 
level is likely to be populated at some temperature. If the level 
is much higher than k B T, then it is unlikely to be occupied. 
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Worked example 



Example 8 

There is one mole of atoms at a temperature of 1 K. Each 
atom has energy levels — 10 -23 J and 10 -23 J. Find the number 
of atoms at each level. 

Solution 

The formula is Boltzmann distribution 

rii = Aexp(-Si/k B T). 

Given 

e 1 = -1CT 23 J, s 2 = 1CT 23 J, and T = 1 K. 

We also know that ni + n2 = (Avogadro constant). 

We need to find the numbers n\ and n^. But what is A? 
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Lets write out all equations: 

ni + n 2 = N A 



ni = Aexp(-si/k B T) 

ri2 = Aexp(-S2/k B T) 

There are three unknowns: ni, n 2 and A. Divide the last two 
equations: 

Ti 1 

— = exp((e 2 -ei)/feBT). 
ri2 

Let this above fraction be r. So the total is N A is divided into 
the ratio r : 1. Therefore 

ni = — ^- X A^4 = 4.876 x 10 23 
r + 1 

n 2 = xiV A = 1.145 x 10 23 

r + 1 
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Worked example 



Example 9 

Following from the previous example, find the total energy of 
the atoms. 

Solution 

There are n\ atoms with energy e lf so the energy of these 
atoms is n\e\ 

There are atoms with energy s 2 - so the energy of these 
atoms is n^e^. 

Therefore the total energy is 

U = n\E\ + 77-262- 

Using the values from the previous example, we get 

U = (4.876 x 10 23 )(-10~ 23 ) + (1.145 x 10 23 )(10~ 23 ) = 6.021J. 

58 



Worked example 



Example 10 



Following from the previous example, find the probability that 
an atom is at level 1, and the probability that it is at level 2. 

Solution 

The total number of atoms is N^. 

There are n\ atoms at level 1 and atoms at level 2. 

An atom can change level with time. So any one time, 
probability that it is at level 1 is 

number of possible atoms at level 1 _ n\ _ 4.876 x 10 23 

Na ~ 6.022 x 10 23 

n 2 _ 1.145 x 10 23 
total number of atoms 6.022 x 10 23 



total number of atoms 
Probability that it is at level 2 is 

number of possible atoms at level 2 
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Worked example 



Example 11 

There are a number of atoms at a temperature of 1 K. Each 
atom has energy levels — 10 -23 J and 10 -23 J. Find the 
probability that an atom is at each level. 

Solution 

This time, we are not given the total number of particles. We 
still have the Boltzmann distribution: 

ni = Aexp(-si/k B T) 

ri2 = Aexp(-S2/k B T) 

but we cannot find A if we don't know the the total number of 
particles. 
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Worked example 



But we know that the probability that an atom is at level 1 is 

number of possible atoms at level 1 _ n\ 

total number of atoms n\ + n 2 

If we substitute the Boltzmann distribution, A cancels out! 

ni XexpC-gi/fcffT) 

n 1 + n 2 ~ Aexp(-s 1 /k B T) + Aexp(-s 2 /k B T) 

We do know the energy levels and temperature, so we can not 
calculate this. In the same way, the probability that an atom is 
at level 2 is 

n 2 Aexp(-s 2 /k B T) 

n 1 +n 2 ~ Xexp(-si//c B T) + Aexp(-e 2 /k B T) 

The answers are 4.876/6.022 and 1.145/6.022 respectively. 
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